te  technical  note  tech 


Operational  Test  and  Evaluation 
(OT&E)Test  Plan  for  the  Federal 
Aviation  Administration  (FAA)  Bulk 
Weather  Telecommunications 
Gateway  (FBWTG) 


William  E.  Benner 
Bharat  Khatiwala 


January  1999 
DOT/FAA/CT-TN99/2 


Document  is  on  file  at  the  William  J.  Hughes  Technical  Center 
Library,  Atlantic  City  International  Airport,  NJ  08405 


© 

U.S.  Department  of  Transportation 

Federal  Aviation  Administration 

William  J.  Hughes  Technical  Center 
Atlantic  City  International  Airport,  NJ  08405 

DISTRIBUTION  STATEMENT  A 
Approved  for  Public  Release 
dtic  quality  inspected  4  Distribution  Unlimited 


19990330  103 


NOTICE 


This  document  is  disseminated  under  the  sponsorship 
of  the  U.S.  Department  of  Transportation  in  the  interest  of 
information  exchange.  The  United  States  Government 
assumes  no  liability  for  the  contents  or  use  thereof. 

The  United  States  Government  does  not  endorse 
products  or  manufacturers.  Trade  or  manufacturers’ 
names  appear  herein  solely  because  they  are  considered 
essential  to  the  objective  of  this  report. 


Technical  Report  Documentation  Page 


1.  Report  No.  2.  Government  Accession  No. 

DOT/FAA/CT-TN99/2 

3.  Recipient’s  Catalog  No. 

4.  Title  and  Subtitle 

5.  Report  Date 

Operational  Test  and  Evaluation  (OT&E)  Test  Plan  for  the  Federal 

January  1999 

Aviation  Administration  (FAA)  Bulk  Weather  Telecommunications 

Gateway  (FBWTG) 

6.  Performing  Organization  Code 

7.  Author(s) 

8.  Performing  Organization  Report  No. 

Bart  Khatiwala,  William  E.  Benner;  Gerald  DiMassa,  BCI 

DOT/FAA/CT-TN99/2 

9.  Performing  Organization  Name  and  Address  10.  Work  Unit  No  (TRAILS) 

U.S.  Department  of  Transportation 

Federal  Aviation  Administration  _ _ 

William  J.  Hughes  Technical  Center  11.  Contract  or  Grant  No. 

Atlantic  City  International  Airport,  NJ  08405 _ 

12.  Sponsoring  Agency  Name  and  Address  13.  Type  of  Report  and  Period  Covered 

U.S.  Department  of  Transportation  Technical  Note 

Federal  Aviation  Administration  _ 

William  J.  Hughes  Technical  Center  14.  Sponsoring  Agency  Code 

Atlantic  City  International  Airport,  NJ  08405 _ | _ 

15.  Supplementary  Notes 

16.  Abstract 

The  Federal  Aviation  Administration  (FAA)  Bulk  Weather  Telecommunications  Gateway  (FBWTG)  is  a  communications 
gateway  to  the  FAA  for  high-resolution  gridded  weather  forecast  data  (e.g.,  Eta  Forecast  Model,  Aviation  (AVN)  and 
Rapid  Update  Cycle  (RUC)  and  airborne  observation  data  (e.g.,  Meteorological  Data  Collection  and  Reporting  System 
(MDCRS))  from  the  National  Weather  Service  Telecommunications  Gateway  (NWSTG).  The  FBWTG  will  provide  the 
capability  to  simultaneously  and  continuously  receive  high-resolution  gridded  weather  forecast  data  and  MDCRS  data 
from  the  NWSTG  and  disseminate  this  information  to  other  National  Airspace  System  (NAS)  subsystems. 

The  purpose  of  this  plan  is  to  provide  an  overview  of  the  Operational  Test  and  Evaluation  (OT&E)  phase  of  System 
Testing  for  both  pathways;  NWSTG  to  FBWTG  located  at  the  Air  Traffic  Control  System  Command  Center 
(ATCSCC),  and  FBWTG  to  National  Weather  Service  (NWS)  Filter  Unit  (NFU)-Test  device  located  at  the  William  J. 
Hughes  Technical  Center.  To  accomplish  this  objective,  the  FBWTG,  an  end-state  subsystem,  must  be  operationally 
integrated  with  NAS  subsystems.  This  plan  establishes  test  strategy,  test  objectives,  test  criteria,  and  detailed  test  and 
evaluation  descriptions  to  be  used  in  the  generation  of  the  Test  Procedures  and  Test  Report’s  entrance  and  exit  criteria 
for  the  program. 

17.  Keywords  18.  Distribution  Statement 

FAA  Bulk  Weather  Telecommunications  Gateway  (FBWTG)  Document  is  on  file  at  the  William  J.  Hughes 
National  Weather  Service  Telecommunications  Gateway  Technical  Center  Library,  Atlantic  City  International 
(NWSTG)  Airport,  NJ  08405 

Meteorological  Data  Collection  and  Reporting  System 
(MDCRS);  Rapid  Update  Cycle  (RUC);  Aviation  (AVN) 

Eta;  NWS  Filter  Unit  (NFU) _ _ 


19.  Security  Classif. 

20.  Security  Classif.  (of  this  page) 

21.  No.  of 

22.  Price 

(of  this  report) 

pages 

Unclassified 

Unclassified 

47 

Form  DOT  F  1700.7  (8-72)  Reproduction  of  completed  page  authorized 


TABLE  OF  CONTENTS 


Page 

EXECUTIVE  SUMMARY  v 

1.  INTRODUCTION  1 

1.1  Background  1 

1 .2  Purpose  1 

1 .3  Scope  2 

2.  REFERENCE  DOCUMENTS  2 

2.1  FAA  Documents  2 

2.2  Other  Documents  2 

3.  SYSTEM  DESCRIPTION  3 

3.1  System  Overview  3 

3 .2  Interfaces  3 

3.3  Activities  Leading  to  OT&E  Testing  5 

4.  TEST  PROGRAM  DESCRIPTION  6 

4. 1  Test  Approach  6 

4.2  Test  and  Analysis  Tools  10 

4.3  Test  and  Evaluation  Descriptions  1 1 

5.  TEST  MANAGEMENT  22 

5.1  Test  Management  Organization  22 

5.2  System  Configuration  Management  22 

5.3  OT&E  Test  Entry  Criteria  23 

5 .4  Test  Execution  23 

5.5  OT&E  Test  Exit  Criteria  23 

5.6  Operational  Test  Reports  24 

5.7  Trouble  Reporting  24 

5.8  Operational  Test  Schedule  24 

5.9  Planning  Considerations  and  Limitations  25 

6.  VERIFICATION  METHODS  25 


iii 


7.  ACRONYMS  AND  ABBREVIATIONS 


26 


APPENDIX 


A  Verification  Requirements  Traceability  Matrix 
B  OT&E  Test  Evaluation  Matrix 
C.  Problem  Trouble  Report 


LIST  OF  ILLUSTRATIONS 


Figure 

Page 

3.1-1 

FBWTG  Gateway  Overview 

3 

4.1. 1-1 

Test  Objectives 

7 

5.2. 1-1 

Test-Bed  Configuration 

LIST  OF  TABLES 

23 

Table 

Page 

3.2.1.2.1-1 

File  Dissemination 

4 

4.1. 1.2-1 

MOE,  MOS,  and  MOP  Definitions 

8 

4.1.1. 3.2-1 

Message  /  File  Loading 

10 

4.2-1 

NFU  Test  Device 

11 

5.1-1 

Test  Management  Organization 

22 

5.8-1 

Operational  Test  Schedule 

25 

IV 


EXECUTIVE  SUMMARY 


The  Federal  Aviation  Administration  (FAA)  Bulk  Weather  Telecommunications  Gateway  (FBWTG) 
adds  the  capability  for  the  Air  Traffic  Control  System  Command  Center  (ATCSCC)  Weather  and  Radar 
Processor  (WARP)  System  to  acquire  high  resolution  gridded  data  from  the  National  Weather  Service 
(NWS)  Telecommunications  Gateway  (NWSTG),  and  disseminate  it  to  the  WARP  Systems  and  the 
Integrated  Weather  System  (ITWS)  National  Weather  Service  Filter  Unit  (NFU). 

Formal  Operational  Test  and  Evaluation  (OT&E)  will  be  conducted  by  the  government  at  the  ATCSCC, 
located  in  Herndon,  VA,  upon  successful  completion  of  Harris  Corporation's  Factory  Acceptance  Test 
(FAT)  at  the  contractor's  site.  Data  will  be  disseminated  to  the  NFU-Test  device  at  the  William  J.  Hughes 
Technical  Center,  Atlantic  City  International  Airport,  for  validation  of  the  ITWS/FBWTG  functionality. 

This  Test  Plan  describes  all  testing  to  be  conducted  by  the  Integrated  Product  Team  (IPT)  to  determine 
whether  Acquisition  Program  Baseline  (APB)  requirements  have  been  met,  whether  Operational  Issues 
(01)  have  been  resolved,  and  whether  the  new  system  is  suitable  for  use  in  the  National  Airspace  System 
(NAS). 
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1.  INTRODUCTION. 


The  Federal  Aviation  Administration  (FAA)  Bulk  Weather  Telecommunications  Gateway  (FBWTG)  is 
added  as  a  new  subsystem  to  the  Air  Traffic  Control  System  Command  Center  (ATCSCC)  Stage  1 
Weather  and  Radar  Processor  (WARP)  System  to  acquire  high  resolution  forecast  data  and  airborne 
observation  messages  from  the  National  Weather  Service  Telecommunications  Gateway  (NWSTG)  and 
disseminate  it  to  the  WARP  Systems  and  the  Integrated  Weather  Systems  (ITWS)  National  Weather 
Service  (NWS)  Filter  Unit  (NFU). 

Formal  Operational  Test  and  Evaluation  (OT&E)  will  be  conducted  by  the  government  at  the  ATCSCC, 
located  in  Herndon,  VA,  upon  successful  completion  of  Harris  Corporation's  Factory  Acceptance  Test 
(FAT)  at  the  contractor's  site.  Data  will  be  disseminated  to  the  NFU-Test  device  at  the  William  J.  Hughes 
Technical  Center  for  validation  of  the  ITWS/FBWTG  functionality. 

This  Test  Plan  describes  all  testing  to  be  conducted  by  the  Integrated  Product  Team  (IPT)  to  determine 
whether  Acquisition  Program  Baseline  (APB)  requirements  have  been  met,  whether  Operational  Issues 
(01)  have  been  resolved,  and  whether  the  new  system  is  suitable  for  use  in  the  National  Airspace  System 
(NAS). 

1.1  BACKGROUND. 

The  FAA  has  identified  the  need  for  FBWTG,  which  is  a  subsystem  of  the  WARP  System  located  at  the 
ATCSCC. 

The  purpose  of  the  FBWTG  is  to  serve  as  an  FAA  gateway  for  high-resolution  gridded  weather  forecast 
data  (e.g.,  Eta  Forecast  Model,  Aviation  (AVN)  and  Rapid  Update  Cycle  (RUC)  and  airborne 
observation  data  (e.g.,  Meteorological  Data  Collection  and  Reporting  System  (MDCRS))  from  the 
NWSTG.  The  FBWTG  will  provide  the  capability  to  simultaneously  and  continuously  receive  high- 
resolution  gridded  weather  forecast  data  and  MDCRS  data  from  the  NWSTG  and  disseminate  this 
information  to  other  NAS  subsystems. 

1.2  PURPOSE. 


The  OT&E  testing  is  performed  to  determine  the  Operational  Suitability  and  Effectiveness  of  the 
FBWTG  system  under  operationally  realistic  conditions.  The  purpose  of  this  plan  is  to  provide  an 
overview  of  the  OT&E  phase  of  system  testing  for  both  pathways;  NWSTG  to  FBWTG  located  at  the 
ATCSCC,  Herndon,  VA,  and  FBWTG  to  NWS  NFU-Test  device  located  at  the  Technical  Center.  To 
accomplish  this  objective,  the  FBWTG,  an  end-state  subsystem,  must  be  operationally  integrated  with 
NAS  subsystems.  This  plan  establishes  test  strategy,  test  objectives,  test  criteria,  and  detailed  test  and 
evaluation  descriptions  to  be  used  in  the  generation  of  the  Test  Procedures  and  Test  Report's  entrance 
and  exit  criteria  for  the  program. 

System  Test  consists  of  Development  Test  and  Evaluation  (DT&E),  Contractor  Site  Acceptance  Test 
(SAT)  and  FAA  OT&E.  The  Test  Plan,  described  in  this  document,  and  the  follow-on  procedures  will  be 
prepared  and  approved  to  be  consistent  with  the  Acquisition  Management  System  (AMS)  Test  and 
Evaluation  (T&E)  Process  Guidelines.  The  DT&E  and  SAT  Test  Plans  will  be  the  responsibility  of  the 
contractor  and  witnessed  by  ACT-320. 
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1.3  SCOPE. 


The  Operational  Test  Plan  (OTP)  forms  the  basis  for  the  Operational  Test  (OT)  Program.  This  Test  Plan 
describes  all  testing  to  be  conducted  by  the  IPT  to  determine  whether  APB  requirements  have  been  met, 
whether  OIs  have  been  resolved,  and  whether  the  new  system  is  suitable  for  use  in  the  NAS. 

Three  major  components  of  the  OTP  are  described:  NAS  Integration,  Operational  Suitability,  and 
Operational  Effectiveness.  The  OTP  also  describes  the  various  planning  and  preparation  activities 
required  prior  to  OT,  the  operational  testing  strategy/approach,  test  resource  requirements,  and  test  entry 
conditions.  This  Plan  contains  an  OT  schedule  and  a  Verification  Requirements  Traceability  Matrix 
(VRTM)  (appendix  A).  Testing  of  the  internal  interfaces  to  the  WARP  subsystem  is  beyond  the  scope 
of  this  Test  Plan,  and  will  be  tested  during  the  DT&E  phase.  Thus,  verification  of  the  interface  between 
the  NWS  AVAR?  will  not  be  addressed  in  this  Test  Plan. 


Formal  OT&E  will  be  conducted  by  the  government  at  the  ATCSCC,  located  in  Herndon,  VA,  upon 
successful  completion  of  Harris’s  Factory  Acceptance  Test  (FAT)  at  the  contractor's  site.  Data  will  be 
disseminated  to  the  NFU-Test  device  at  the  Technical  Center  for  validation  of  the  ITWS/FBWTG 
functionality.  More  detail  of  the  OT&E  test  program  is  provided  in  section  4  of  this  document. 

2.  REFERENCE  DOCUMENTS. 

The  following  specifications,  standards,  and  other  documents  form  a  part  of  this  Test  Plan. 

2.1  FAA  DOCUMENTS. 

FAA-E-2898  Weather  and  Radar  Processor  (WARP)  System  Specification 

FAA-G-2100F  Electronic  Equipment,  General  Requirements 


2.2  OTHER  DOCUMENTS. 

AMS  Acquisition  Management  System  (AMS)  Test  and  Evaluation  (T&E) 

Process  Guidelines,  dated  February  1999 


NAS-IC-XXXXXXXX  FAA  Bulk  Weather  Telecommunications  Gateway  (FBWTG)  to 

WARP  Interface  Control  Document  (ICD) 


NAS-IR-900294 1 4 


NAS-IR-94 142514 


NAS-XX-XXXXX 


Interface  Requirements  Document  (IRD)  for  National  Weather 
Service  Telecommunications  Gateway  (NWSTG)  to  Federal  Aviation 
Administration  Bulk  Weather  Telecommunications  Gateway 
(FBWTG) 

Interface  Requirements  Document  (IRD)  for  Federal  Aviation 
Administration  Bulk  Weather  Telecommunication  Gateway 
(FBWTG)  to  Integrated  Terminal  Weather  System  (ITWS) 

FAA  Bulk  Weather  Telecommunications  Gateway  (FBWTG) 
Operational  Requirements  Document  (ORD),  dated  August  14,  1997 


2 


WARP  DTFA01-96-C-00036  FAA  Bulk  Weather  Telecommunications  Gateway  (FBWTG) 

Subsystem  Statement  of  Work  (SOW) 

3.  SYSTEM  DESCRIPTION. 

3.1  SYSTEM  OVERVIEW. 

Initially,  the  FBWTG  will  acquire  high-resolution  gridded  weather  forecast  data  and  airborne 
observation  data  messages  from  the  NWSTG  and  disseminate  the  data  to  the  Integrated  Terminal 
Weather  System  (ITWS)  and  WARP.  The  FBWTG  will  serve  as  a  subsystem  of  the  WARP  system 
located  at  ATCSCC. 

Figure  3.1-1  depicts  the  FBWTG  Gateway  Overview.  The  NWSTG  provides  the  FBWTG  with  files 
containing  messages  that  provide  forecast  information  and  meteorological  observations.  These  messages 
are  accumulated  by  the  NWSTG  over  a  preset  period  of  time,  sorted  by  product  type,  and  transferred  to 
the  FBWTG.  The  FBWTG  is  responsible  for  dissemination  of  NWS  provided  messages  to  the  ITWS 
NFU  and  WARP. 


Legend: 

Message  Flow 


NWS 

(Silver  Spring,  MD) 


NWSTG 


WARP 


ATCSCC 

(Herndon,  VA) 


ATCSCC 

WARP 

FBV 

VTG 

J  V 


WJHTC 


NFU-  Test 
Device 


FIGURE  3.1-1 .  FBWTG  GATEWAY  OVERVIEW 


3.2  INTERFACES. 

The  demarcation  points  and  protocols  used  between  the  NWSTG  and  the  FBWTG  are  defined  in  the 
NWSTG  to  FBWTG  Interface  Requirements  Document  (IRD),  NAS-IR-90029414.  The  demarcation 
points  and  protocols  used  between  the  FBWTG  and  ITWS  are  defined  in  the  FBWTG  to  ITWS  IRD, 
NAS-IR-94142514.  The  demarcation  points  and  protocols  used  between  the  FBWTG  and  WARP  are 
defined  in  the  FBWTG  to  WARP  ICD. 

3.2.1  Operational  Modes  Overview. 

The  operational  modes  of  the  FBWTG  consists  of  the  following: 
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a.  Acquisition  Mode 

b.  Dissemination  Mode 

c.  Data  management  Mode 

d.  System  Monitoring  and  Control  Mode 

3.2.1.1  Acquisition  Mode. 

The  FBWTG  will  acquire  weather  files  containing  World  Meteorological  Organization  (WMO) 
messages  from  the  NWSTG  as  specified  in  the  NWSTG/FBWTG  IRD. 

3. 2.1.2  Dissemination  Mode. 

The  FBWTG  will  automatically  disseminate  acquired  files  to  ITWS  as  specified  in  the  FBWTG/ITWS 
IRD;  and  WMO  messages  to  WARP  as  specified  in  WARP/FBWTG ICD  To  Be  Determined  (TBD). 

3. 2 .1.2.1  Performance. 

The  FBWTG  will  meet  the  performance  requirements  for  dissemination  of  files  to  ITWS  and  WMO 
messages  to  WARP: 


TABLE  3 .2. 1.2. 1-1.  FILE  DISSEMINATION 


INTERFACE 

AVERAGE 

99.5  PERCENTILE 

99.95  PERCENTILE 

ITWS 

3  seconds 

8  seconds 

10  seconds 

WARP 

3  seconds 

8  seconds 

10  seconds 

•  Time  is  measured  from  notification  of  receipt  of  the  last  bit  of  the  NWS  file  until 
the  first  bit  of  the  data  to  be  disseminated  is  placed  on  the  NAS  subsystem 
interface. 

•  The  FBWTG  will  disseminate  an  NWS  file/WMO  message  to  the  respective 
interfaces  within  30  seconds  of  receipt  of  the  file  from  the  NWSTG. 

3. 2.1.3  Data  Management  Mode. 

The  FBWTG  will  retain: 

a.  The  past  two  model  runs  for  the  AVN  and  Eta  model; 

b.  The  past  two  model  runs  of  both  the  1 -hourly  and  3-hourly  RUC  model; 

c.  One  hour  of  MDCRS  data. 

The  FBWTG  periodically  purges  the  database  of  the  oldest  data,  while  ensuring  retention  of  data  as  listed 
above. 
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The  FBWTG  maintains  a  reception  log  containing  a  separate  entry  for  each  file  that  has  been  received 
from  the  NWSTG.  These  logs  include;  data  type,  model  run  (if  file  contains  high-resolution  gridded 
weather  data)  or  hour  in  which  MDCRS  observations  were  taken,  and  hour/minute  in  which  the  file  was 
pushed  to  the  FBWTG.  The  FBWTG  maintains  a  dissemination  log  containing  a  separate  entry  for  each 
file  that  has  been  disseminated  to  external  destinations.  These  logs  include;  data  type,  model  run  (if  file 
contains  high-resolution  gridded  data)  or  hour  in  which  MDCRS  observations  were  taken,  and 
hour/minute  in  which  the  file/WMO  message  data  was  pushed  or  transmitted  to  WARP  and  ITWS. 

3.2. 1.4  System  Monitoring  and  Control  Mode. 

The  FBWTG  maintains  a  system  alert  log  containing  a  separate  entry  for  each  system  alert. 

For  each  event,  the  FBWTG  system  alert  log  contains  the  Universal  Coded  Time  (UTC)  for  detection, 
specific  nature  of  the  event,  identification  of  the  originator,  user  or  component,  and,  if  applicable, 
additional  diagnostic  data. 

The  FBWTG  maintains  a  resource  utilization  log  containing  the  percentage  of  system  resources  used, 
averaged  over  1 5  minutes,  and  measured  every  1  minute  for  the  following: 

a.  Disk  Utilization 

b.  Memory  Paging  Average 

c.  Internal  Communication  Bandwidth 

d.  Central  Processing  Unit  (CPU)  Utilization 

The  FBWTG  system  logs  are  available  for  display  as  follows: 

a.  All  and/or  selected  portions  of  the  system  logs 

b.  Current  logs  as  they  are  produced 

3.3  ACTIVITIES  LEADING  TO  OT&E  TESTING. 

FAT  will  be  conducted  by  Harris  Corporation  at  the  contractor's  site  and  witnessed  by  the  FAA.  Harris 
Corporation  will  provide  a  T&E  plan  that  will  ensure  that  the  FBWTG  subsystem  is  compliant  with  the 
specification  requirements. 

Harris  Corporation  will  complete  verification  of  the  remaining  FBWTG  requirements  during  SAT  at  the 
Herndon,  VA  site.  Harris  Corporation  will  also  conduct  a  series  of  system  tests  to  verify  the 
requirements  that  have  not  been  previously  verified  and  conduct  appropriate  regression  tests.  The  SAT 
will  be  conducted  using  government-approved  Harris  Corporation  furnished  scenarios  and  a  system 
configuration  adapted  with  ATCSCC  parameters. 

Harris  Corporation  will  also  deliver  and  install  a  fully  integrated  configuration  for  the  Herndon,  VA  site. 
Harris  will  also  verify  that  the  subsystem  is  fully  operational;  is  tailored  to  specific  site  requirements;  has 
complete  documentation,  tools,  and  test  equipment;  and  is  complete  and  ready  for  government  operation 
and  maintenance. 

OT&E  Testing  will  be  conducted  at  the  Herndon,  VA  site  using  the  test-bed  configuration  described  in 
paragraph  5.2.1  with  support  from  NWS  (Silver  Spring,  MD)  and  Harris  Corporation. 
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4.  TEST  PROGRAM  DESCRIPTION. 


The  FBWTG  OT&E  test  effort  will  ensure  that  the  FBWTG  Subsystem  is  Operationally  Effective  and 
Operational  Suitable  and  is  able  to  be  integrated  into  the  NAS.  Operational  Effectiveness  centers  around 
the  performance  of  the  FBWTG  System,  while  Operational  Suitability  centers  on  the  maintainability  of 
the  FBWTG  System. 

4.1  TEST  APPROACH. 

The  test  will  be  run  on  operational  hardware  and  the  baseline  software.  All  major  functions  and 
interfaces  will  be  exercised  during  this  test  phase.  Each  test  case  is  traceable  to  a  specific  requirement. 
Failure  to  complete  the  entire  test  case  for  any  reason  requires  the  test  case  to  be  restarted. 

4.1.1  Test  Objectives. 

The  primary  objective  of  OT  is  to  demonstrate  that  a  new  system  is  Operationally  Effective  and  Suitable 
for  use  in  the  NAS,  and  that  the  NAS  infrastructure  is  ready  to  accept  the  system.  These  tests  focus  on 
demonstrating  that  operational  requirements  have  been  met  and  that  all  Operational  Issues  (OIs)  have 
been  resolved.(See  figure  4. 1.1-1.) 

The  major  components  of  OT  are  NAS  Integration,  Operational  Suitability,  and  Operational 
Effectiveness  tests,  which  are  addressed  in  the  following  sections. 

•  NAS  Integration  testing  verifies  that  the  system  interfaces  to  the  existing  elements  of  the  NAS  and 
that  the  NAS  can  operate  with  the  new  subsystem. 

•  Operational  Suitability  testing  evaluates  the  degree  to  which  a  product  intended  for  field  use 
satisfies  its  availability,  interoperability,  reliability,  maintainability,  human  factors,  logistics,  and 
documentation. 

•  Operational  Effectiveness  testing  evaluates  the  degree  to  which  a  product  accomplishes  its  mission 
when  used  by  representative  personnel  in  the  expected  operational  environment.  This  testing 
includes  capacity  and  NAS  loading,  degraded  mode  operations,  security,  and  planned  life  cycle. 

OT  effectiveness  testing  also  assesses  OIs. 
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FIGURE  4. 1.1-1.  TEST  OBJECTIVES 


4.1. 1.1  NAS  Integration. 

The  objectives  of  NAS  Integration  Testing  are  defined  below. 

4.1 .1 .1 .1  NAS  Subsystem  and  System  Integration  Test. 

The  objective  of  this  test  is  to  verify  the  interfaces  and  product  exchanges  between  the  FBWTG, 
NWSTG  and  the  NFU  subsystems.  This  test  is  comprised  of  four  test  cases: 

a.  Acquisition  Mode  Test  Case:  File  Transfer  Protocol  (FTP)  and  file  registry 

b.  Dissemination  Mode:  FTP,  product  ordering  and  delivery  time,  and  purge 

c.  Data  Management  Mode:  purging,  data  retention,  and  operations  log 

d.  System  Monitoring  and  Control  Mode:  support  concept 

4. 1.1. 1.2  NAS  Interface  Test. 

The  objective  of  this  test  is  to  verify  the  requirements  specified  in  the  NWS  /  FBWTG  and  FBWTG  / 
ITWS  IRDs.  This  test  is  comprised  of  one  test  case: 

a.  Interface  Test  (NOTE:  This  does  not  include  the  interface  between  the  FBWTG  /  WARP.) 

4. 1.1 .2  Operational  Suitability. 

The  objective  of  Operational  Suitability  Testing  is  to  evaluate  the  degree  to  which  the  FBWTG  intended 
for  field  use  satisfies  the  requirements  defined  below.  Test  responses  and  comments  to  the  OIs  will  be 
recorded  by  the  test  team  on  an  OT&E  Test  Evaluation  Matrix  (appendix  B)  based  on  the  Measure  of 
Effectiveness  (MOE),  Measure  Of  Suitability  (MOS),  and  Measure  Of  Performance  (MOP)  Definitions 
(table  4.1. 1.2-1). 
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TABLE  4. 1 . 1 .2- 1 .  MOE,  MOS,  AND  MOP  DEFINITIONS 


MEASURE  OF 

EFFECTIVENESS 

(MOE) 

Operational  Effectiveness  addresses  the  degree  to  which  a  product 
accomplishes  its  mission  when  used  by  representative  personnel  in  the 
expected  operational  environment. 

MEASURE  OF 
SUITABILITY 
(MOS) 

Operational  Suitability  addresses  the  degree  to  which  a  product  intended  for 
field  use  satisfies  its  availability,  reliability,  maintainability,  safety,  and 
logistics. 

MEASURE  OF 

PERFORMANCE 

(MOP) 

Quantitative/qualitative  values  which  characterize  the  MOEs/MOSs.  These 
values  are  measurable  by  a  test  process. 

4.1. 1.2.1  Reliability  and  Availability  Test. 

This  test  will  be  conducted  to  estimate  and/or  verify  that  the  Reliability,  Maintainability  and  Availability 
(RMA)  requirements  are  achievable  in  an  operational  environment.  RMA  issues  include  Mean  Time  to 
Repair  (MTTR),  Mean  Time  Between  Failures  (MTBF)  and  FBWTG  Availability  and  Reliability.  In 
general,  the  RMA  requirements  cannot  be  fully  tested  during  OT&E.  Instead,  data  will  be  collected  and 
retained  during  DT&E  and  OT&E  to  support  analysis  of  the  RMA  performance. 

4. 1.1 .2 .2  Maintainability. 

This  test  case  will  be  conducted  to  validate  the  FBWTG’s  capability  to  identify  and  diagnose  system 
faults  and  procedures  to  restore  system  operation  via  specific  hardware  and  software  maintenance  tests. 

4.1.1.2.3  Human  Factors. 

This  test  case  will  be  conducted  to  evaluate  operation  and  maintenance  tasks  of  the  FBWTG  via 
Applicable  Human  Factors  (HF)  performance  selected  from  the  following  list: 

a.  FBWTG  Input/Outout  (I/O')  Devices:  System  I/O  display  criteria  adheres  to  the  following: 

1 .  Menu  Displays  -  Menus  are  well  organized  and  are  an  aid  to  the  maintenance 
technicians. 

2.  Status  Displays  -  All  pertinent  data  is  displayed  logically  and  in  an  easy  to  use 
format. 

b.  Alarms/ Alerts:  Alarms/ Alerts  are  adequate  to  attract  attention  but  not  unnecessarily 
disruptive.  Status  lights,  which  indicate  proper  system  status,  are  readily  and  easily 
understood. 

c.  Error  Detection:  Input  parameter  limitations  cannot  be  exceeded  (e.g.,  incorrect  values  are 
detected  and  flagged,  or  rejected). 

d.  Safety:  The  system  will  be  accessed  to  assure  that  it  does  not  present  any  safety  hazards  to 
the  user,  the  facility,  or  other  interface  systems. 
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4.1 .1.2.4  Loeistics. 


This  test  case  will  be  conducted  to  provide  an  overall  assessment  of  the  adequacy  of  the 
logistics  support  for  Operational  Suitability  Issues: 

a.  Expediency:  Spare  parts,  etc.,  can  be  obtained  in  a  reasonable  time. 

b.  Turn-Around:  The  system  meets  MTTR  requirements. 

4.1 .1.2.5  Documentation. 

This  test  case  will  be  conducted  to  evaluate  documents  against  the  applicable  test  issues  selected  from 
the  following  documentation  items: 

a.  Completeness:  Sufficient  documentation  is  provided  to  allow  a  working  knowledge  of  the 
FBWTG. 

b.  Readability:  Procedures  and  descriptions  are  written  in  an  easy  to  understand  manner. 

c.  Usability:  Reference  documentation  is  logically  organized  and  easy  to  use. 

d.  Revisions:  Contractor  and/or  COTS  manuals,  and  all  other  documents  will  be  assessed  to 
assure  that  standard  update  procedures  are  followed  and  can  be  adequately  maintained. 

4.1 .1.3  Operational  Effectiveness. 

The  objective  of  Operational  Effectiveness  Testing  is  to  evaluate  the  degree  to  which  the  FBWTG 
accomplishes  its  mission  when  used  by  representative  personnel  in  the  expected  operational 
environment. 

4. 1 . 1 .3 . 1  Degraded  Mode  Operations  Test. 

The  objective  of  this  test  is  to  evaluate  the  ability  of  the  FBWTG  to  tolerate  and  respond  to  circuit 
failures  and  outages.  Based  on  the  NWSTG,  NFU  outages,  and  link  loss,  product  requests  will  be 
utilized  to  restore  operations  with  minimum  impact.  This  test  is  comprised  of  four  test  cases: 

a.  Failure  Mode 

b.  Interface  Failure  Detection 

c.  Alert  Notification  and, 

d.  Return  to  Normal  Notification 

4. 1.1 .3.2  NAS  Loading. 

The  objective  of  this  test  is  to  validate  the  NAS  Loading  performance  by  measuring  the  file 
dissemination  time  (see  table  4.1. 1.3 .2-1). 
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TABLE  4. 1 . 1 .3.2-1 .  MESSAGE  /  FILE  LOADING 


INTERFACE 

LOAD 

AVERAGE 

99.5  PERCENTILE 

99.95  PERCENTILE 

ITWS 

TBD 

( messages/hour) 

3  seconds 

8  seconds 

1 0  seconds 

WARP 

TBD 

(messages/hour) 

3  seconds 

8  seconds 

10  seconds 

TABLE  4.1.1 .3.2-1 .  MESSAGE  /  FILE  LOADING 

a.  Time  is  measured  from  notification  of  receipt  of  the  last  bit  of  the  NWS  file 
until  the  first  bit  of  the  data  to  be  disseminated  is  placed  on  the  NAS 
subsystem  interface. 

b.  The  FBWTG  will  disseminate  an  NWS  file/WMO  message  to  the  respective 
interfaces  within  30  seconds  of  receipt  of  the  file  from  the  NWSTG. 

4. 1 . 1 .3 .3  Securitv/Adaptation  Data. 

The  security  test  will  demonstrate  that  the  FBWTG  subsystem  has  the  necessary  security  procedures  in 
place  for  an  air  traffic  control  (ATC)  situation.  In  an  operational  setting,  these  include  procedures  to 
restrict  access  to  the  workstation  by  requiring  specific  user  log-ins  and  passwords.  Tests  will  verify  that 
operators’  classes  are  specified  and  are  adaptable.  All  security  anomalies  need  to  be  noted,  with  an  audit 
of  all  matters  involving  security  being  maintained,  and  be  accessible  by  authorized  personnel. 

Integrity  of  the  FBWTG,  NWSTG,  and  the  NFU  subsystems  will  be  tested  for  security  issues.  This  will 
include  both  proper  and  improper  Internet  Protocol  (IP)  addresses,  and  FTP  operations.  Passwords  will 
be  used  to  test  the  security  of  the  system  support  functions. 

Adaptation  testing  will  insure  that  the  government  can  modify  an  adaptation  file  using  standard  tools  to 
rebuild  the  program  and  verify  the  changes. 

4.1. 1.3.4  Planned  Life  Cycle. 

Upgradability  of  the  FBWTG  will  be  tested  for  future  product  upgrades  and  users.  This  will  include 
upgrades  from  the  NWS,  National  Center  for  Environmental  Prediction  (NCEP),  and  Aviation  Weather 
Center  (AWC)  product  and  services  upgrades,  and  commercially  available  software  packages. 

4.2  TEST  AND  ANALYSIS  TOOLS. 

Technical  Center  test  labs  will  provide  an  NFU  Test  Device,  System  Monitor  Functions/Tools,  and 
Protocol  Analyzer  to  sufficiently  test  FBWTG  subsystem  interfaces.  The  NFU  Test  Device  capabilities 
are  identified  in  table  4.2-1. 
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TABLE  4.2-1 .  NFU  TEST  DEVICE 


ITEMS 

FUNCTION 

Communications  Subsystem 

Provides  an  external  interface  and  communications 

Data  Reduction  and  Analysis  Tool 

Aids  in  the  analysis  of  messages  and  products  recorded 

SUN  workstation 

Used  to  develop  specific  scenarios  at  Technical  Center 

SUN  server 

Used  to  perform  specific  server  scenarios  at  Technical  Center 

4.2.1  System  Monitor  Functions  /  Tools. 

The  System  Monitor  (SM)  functions  will  be  used  for  NFU  data  recording,  data  analysis,  file  editing,  file 
management  and  system  monitoring,  with  selected  data  saved  from  these  functions  for  further  analysis 
(disk  or  line  printer  listings). 

4.2.2  Protocol  Analyzer. 

The  protocol  analyzer  will  perform  the  following  functions: 

a.  Verily  IRD  Requirements 

b.  Verify  Message  Formats 

c.  Measure  Performance  of  File  Dissemination 

4.3  TEST  AND  EVALUATION  DESCRIPTIONS. 

The  tables  in  this  section  identify  the  Test  Identification  (ID)  numbers,  the  title  of  each  test/evaluation, 
and  description  of  the  objective,  criteria,  approach,  and  duration  of  each  test.  These  test  descriptions  will 
be  used  as  the  basis  for  the  development  of  the  OT&E  procedures.  These  tables  represent  a  subset  of  the 
OT&E  VRTM,  contained  in  appendix  A  of  this  plan. 

4.3.1  Integration  Testing. 

Integration  testing  is  divided  into  two  test  areas.  Testing  will  examine  system  implementation  and 
performance  as  reflected  in  paragraph  4.3. 1.1. 
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NAS  INTEGRATION  TEST/EVALUATION  DESCRIPTIONS 
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4.3.2  Operational  Suitability  Testing. 
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S-03  human  Factors  4. 1.1. 2.3  Test  procedures  will  provide  Test  participants  will  evaluate  NOTE  1 

overall  assessment  of  Human  the  results  of  each  operational 
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5.  TEST  MANAGEMENT. 


5 . 1  TEST  MANAGEMENT  ORGANIZATION. 

The  following  paragraphs  describe  the  structure  and  composition  of  the  FBWTG  OT&E  test 
organization,  and  identify  the  personnel  responsible  for  the  planning  and  conduct  of  the  test  activities. 
Described  herein  are  the  roles  and  responsibilities  of  the  participating  organizations  (table  5.1-1). 

TABLE  5.1-1.  TEST  MANAGEMENT  ORGANIZATION 


ORGANIZATION 

RESPONSIBILITY 

AUA 

Office  of  Air  Traffic 
Systems  Development 

Provides  FBWTG  Product  Team  Leader  for  NAS  acquisition  program. 
Provides  funding  and  support  resources  to  ACT-320  for  T&E  effort. 
Distributes  contractor  test  documents  for  review,  and  returns  comments  to 
contractor.  Reviews  and  co-approves  all  internal  and  external  test 
documents  relating  to  FBWTG.  Provides  FAA  management  with  program 
status. 

ACT-320 

Weather  Branch 

ACT-320  provides  a  Product  Team  Test  Lead  who  acts  as  an  agent  of  the 
FBWTG  T&E  program.  This  includes;  coordinating  tests,  ensuring  that  all 
test  requirements  are  satisfied,  conducting  OT&E,  approval  of  exit  criteria, 
and  assures  that  tests  are  performed  in  accordance  with  approved 
procedures. 

AOS 

Operational  Support 
Service 

Determination  of  Operational  Suitability,  participation  in  OTP 
development,  review  OT&E  Integration  requirements,  plans,  procedures, 
tests,  and  reports.  Participate  in  the  preparation,  review,  and  conduct  of 
OT&E. 

Harris  Corporation 
Support  Service 

Provide  on-site  technical  assistance  and  support  for  the  OT&E  Test  phase. 

5.2  SYSTEM  CONFIGURATION  MANAGEMENT. 

During  DT&E,  Harris  will  conduct  each  test  activity  using  Configuration  Management  (CM)  control 
documented  in  the  government-approved  Program  Management  Plan  (PMP).  The  government  will 
conduct  configuration  audits  in  accordance  with  the  FBWTG  Statement  of  Work  (SOW).  Audits 
commence  before  and  complete  after  successful  completion  of  FAT. 

5.2.1  Test-Bed  Configuration. 

During  OT&E,  the  FAA  will  be  responsible  for  System  CM  of  the  test-bed  environment,  depicted  in 
figure  5.2. 1-1.  The  FAA  will  also  record/maintain  all  documented  anomalies,  deviations,  and  waivers 
generated  during  OT&E. 


22 


Alarms 
Logs  /  Reports 
Status 


FIGURE  5.2. 1-1.  TEST-BED  CONFIGURATION 

5.3  OT&E  TEST  ENTRY  CRITERIA. 

Formal  OT&E  testing  will  begin  pending  successful  completion  of  the  following  activities/conditions: 

a.  Harris  Corporation  has  successfully  completed  FAT  Testing  at  the  contractor  site. 

b.  FAA  OT&E  Test  Plans  and  Procedures  have  been  approved. 

c.  Availability  of  the  NWSTG  and  telecommunications  pathway  at  the  key  site. 

5.4  TEST  EXECUTION. 

Test  execution  will  commence  upon  completion  of  OT&E  dry  runs,  and  verification  of  the  red-lined  test 
procedures. 

All  collected  data  will  be  compiled  and  analyzed.  Where  applicable,  system/operational  trouble  reports 
will  be  generated  to  document  all  anomalies/discrepancies  identified  during  OT&E.  Where  necessary, 
retest  will  be  conducted  to  verify  any  fixes  implemented  to  address/resolve  system/operational 
deficiency  reports  prior  to  FAA  approval/closure. 

5.5  OT&E  TEST  EXIT  CRITERIA. 

Completion  of  OT&E  testing  depends  on  evaluation  of  the  identified  system  operational  requirements, 
satisfactory  evaluation  of  suitability  issues,  and  the  resolution  of  anomalies/issues  identified  during 
testing. 

A  key  element  of  the  OT&E  Test  Plan  is  the  exit  criteria; 
a.  All  test  scripts  have  been  executed. 
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b.  All  Program  Trouble  Reports  (PTRs)  must  have  been  satisfactorily  resolved,  with 
government  agreement  on  the  resolution 

c.  All  changes  made  as  a  result  of  PTRs  have  been  tested. 

d.  All  documentation  associated  with  the  software  must  be  updated  to  reflect  changes  made 
during  OT&E  testing. 

e.  The  Test  Rreport  has  been  reviewed  and  approved. 

5.6  OPERATIONAL  TEST  REPORTS. 

An  OT&E  Quick  Look  Test  Report  will  be  published  within  10  working  days  after  completion  of  OT&E 
testing  and  a  final  Test  Report  will  be  published  30  calendar  days  after  completion  of  OT&E  testing. 
The  report  will  provide  assessments  and  documented  accounts  of  the  test  activities,  including  execution 
of  test  procedures,  test  results,  data  analysis,  identification  of  participants,  test  schedule,  anomalies, 
deficiencies,  and  deviations. 

5.7  TROUBLE  REPORTING. 

Throughout  the  execution  of  the  OT&E  test  proceedings.  PTRs  will  be  generated  against  the  system 
and/or  program  undergoing  testing  and  evaluation.  All  anomalies/discrepancies  will  be  documented  via 
PTRs,  Review  Item  Discrepancies  (RJDs).  and/or  Action  Items  (AIs).  A  single  form  will  be  used  to 
capture  the  anomalies/discrepancies.  The  form  will  contain  a  check  box  that  will  identify  the  category, 
severity  level  and  priority  of  the  problem  identified.  FAA  approval  of  the  proposed  resolution  strategy, 
implementation,  and  verification  of  the  resolution  is  required  prior  to  FAA  closure  of  the  PTRs. 

5.7.1  Program  Trouble  Reports  (PTR). 

PTRs  are  generated  to  document  any  discrepancies  or  anomalies  encountered  during  OT&E.  These 
records  provide  a  mechanism,  by  which  traceability  and  repeatability  of  the  test  proceedings  are 
maintained.  (See  appendix  C.) 

5.7.2  Review  Item  Discrepancies  (RIDs). 

RIDs  are  generated  to  identify  and  record  any  documentation-related  discrepancies  or  deficiencies 
encountered  during  OT&E. 

5.7.3  Action  Items  (AIs). 

AIs  are  generated  to  document  and  track  issues  and  activities  that  require  completion  and/or  resolution. 

5.8  OPERATIONAL  TEST  SCHEDULE. 

The  FAT  and  OT&E  Test  Schedule  is  as  follows,  see  table  5.8-1. 
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TABLE  5.8-1.  OPERATIONAL  TEST  SCHEDULE 


TASK /ACTIVITY 

DURATION 

START  DATE 

Setup 

1  day  * 

TBD 

Dry  Run 

1  day  * 

TBD 

Test  Conduct 

3  days  * 

TBD 

PTRs,  Quick  Look  Reports, 
Regression  Testing 

1 0  days  * 

TBD 

Final  Report 

30  days  ** 

TBD 

*  Working  days 
**  Calendar  days 


5.9  PLANNING  CONSIDERATIONS  AND  LIMITATIONS. 

Limitations  to  the  scope  of  OT&E  have  not  yet  been  defined. 

6.  VERIFICATION  METHODS. 

The  verification  methods  that  can  be  used  at  any  of  the  three  verification  levels  are  as  follows; 

a.  Inspection  (I)  -  Inspection  is  a  method  of  verification  to  determine  compliance  without  the 
use  of  special  laboratory  equipment,  procedures,  or  services,  and  consists  of  a  non¬ 
destructive  static-state  examination  of  the  hardware,  software,  and/or  the  technical  data  and 
documentation. 

b.  Test  (T)  -  Test  is  a  method  of  verification  wherein  performance  is  measured  during  or  after 
the  controlled  application  of  functional  and/or  environmental  stimuli.  Quantitative 
measurements  are  analyzed  to  determine  the  degree  of  compliance.  The  process  uses 
standardized  laboratory  equipment,  procedures,  and/or  services. 

c.  Demonstration  (D)  -  Demonstration  is  a  method  of  verification  where  qualitative 
determination  of  properties  is  made  for  a  configuration  item,  including  software  and/or  the 
use  of  technical  data  and  documentation.  The  items  being  verified  are  observed,  but  not 
quantitatively  measured,  in  a  dynamic  state. 

d.  Analysis  (AI  -  Analysis  is  a  method  of  verification  where  hardware  or  software  designs  are 
compared  with  known  scientific  and  technical  principles,  procedures,  and  practices  to 
estimate  the  capability  of  the  proposed  design  to  meet  the  mission  and  system  requirements. 

e.  (X)  -  Not  Applicable 
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7.  ACRONYMS  AND  ABBREVIATIONS. 


A 

AI 

AMS 

APB 

ATC 

ATCSCC 

AVN 

AWC 

CM 

D 

DT&E 

Eta 

FAA 

FAT 

FBWTG 

FTP 

HF 

I 

ICD 

ID 

I/O 

IP 

IPT 

IR 


Analysis 
Action  Item 

Acquisition  Management  System 
Acquisition  Product  Baseline 
Air  Traffic  Control 
ATC  System  Command  Center 
Aviation 

Aviation  Weather  Center 
Configuration  Management 
Demonstration 

Developmental  Test  and  Evaluation 

Eta  Forecast  Model  (the  Greek  letter  Eta  is  used  as  the  model  name) 

Federal  Aviation  Administration 

Factory  Acceptance  Testing 

FAA  Bulk  Weather  Telecommunications  Gateway 

File  Transfer  Protocol 

Human  Factors 

Inspection 

Interface  Control  Document 
Identification 
Input  /  Output 
Internet  Protocol 
Integrated  Product  Team 
Interface  Requirements 
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IRD 

Interface  Requirements  Document  ' 

ITWS 

Integrated  Terminal  Weather  System 

LAN 

Local  Area  Network 

MDCRS 

Meteorological  Data  Collection  and  Reporting  System 

MOE 

Measure  of  Effectiveness 

MOP 

Measure  of  Performance 

MOS 

Measure  of  Suitability 

MTBF 

Mean  Time  Between  Failures 

MTTR 

Mean  Time  To  Repair 

NAS 

National  Airspace  System 

NCEP 

National  Centers  for  Environmental  Prediction 

NIMS 

NAS  Infrastructure  Management  System 

NFU 

NWS  Filter  Unit 

NWS 

National  Weather  Service 

NWSTG 

National  Weather  Service  Telecommunications  Gateway 

OI 

Operational  Issue 

OR 

Order 

ORD 

Operational  Requirements  Document 

OT 

Operational  Test 

OT&E 

Operational  Test  and  Evaluation 

OTP 

Operational  Test  Plan 

PMP 

Program  Management  Plan 

PTR 

Program  Trouble  Report 

RID 

Review  Item  Discrepancy 

RMA 

Reliability,  Maintainability,  and  Availability 
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RUC 

SAT 

SM 

SOW 

SPEC 

SRS 

T 

TBD 

T&E 

UTC 

VRTM 

WARP 

WMO 

X 


Rapid  Update  Cycle 
Site  Acceptance  Testing 
System  Monitor 
Statement  of  Work 
Specification 

System  Requirements  Specification 
Test 

To  Be  Determined 
Test  and  Evaluation 
Universal  Coded  Time 

Verification  Requirements  Traceability  Matrix 
Weather  and  Radar  Processor 
World  Meteorological  Organization 
Not  Applicable 
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APPENDIX  A 


VERIFICATION  REQUIREMENTS  TRACEABILITY  MATRIX 


VERIFICATION  REQUIREMENTS  TRACEABILITY  MATRIX 


Paragraph  # 

Requirements) 

Test  ID 

Method 

OPERATIONAL  REQUIREMENTS  NAS-XX-XXXXXXX 

3.2.1 

Operational  Concept 

•  Acquisition  Mode 

•  Dissemination  Mode 

•  Data  management  Mode 

•  System  Monitoring  and  Control  Mode 

1-01, 

S-01  - 
S-02, 

E-01 

D 

Acquisition  Mode  -  FBWTG  shall  acquire  weather  messages  from 
the  NWSTG  as  files  containing  message  sets 

1-01 

D 

2.2.1 

Provide  telecommunications  pathway  between  NWS  and  FA  A 

1-01, 

E-02 

D 

2.2.1. 1 

Provide  adequate  throughput  rate 

1-01 

A 

2.2.2 

FBWTG  Subsystem  shall  acquire  Eta,  AVN,  RUC  gridded  weather 
forecast  data,  and  MDCRS  airborne  weather  observation  data  from 
the  NWSTG. 

1-01 

A,  T,  D 

2.2.2. 1 

Detect  Transmission  Error 

1-01, 

S-01, 

E-01 

T 

2.2.22 

Re-acquire  Data  Following  Transmission  error 

1-01, 

E-01, 

D-01  - 
D-04, 

S-01 

T 

Dissemination  Mode  -  The  FBWTG  shall  disseminate  messages  to 
the  NAS  bulk  weather  recipients 

1-01 

X 

2.2.3 

Provide  Weather  Data  to  WARP  and  ITWS 

1-01 

D 

mm 

Transmit  Data  within  Specified  Time  Frame 

1-01 

T 

2.23.3 

Transmit  Data  in  Specified  Order 

1-01 

A,  T,  D 

2.2.4 

Process  Weather  Data  Re-Transmission  Requests 

1-01 

A,  T,  D 

A-  1 


2.2.4. 1 

Receive  Re-Transmission  Requests 

i-oi 

A,  T,  D 

Evaluate  Re-Transmission  Requests 

1-01 

A 

Re-Acquire  Data  Following  Re-Transmission  Requests 

1-01 

A,  T,  D 

Re-Transmit  Data  to  WARP  and  ITWS 

1-01 

A,  T,  D 

NAS  1000  REQUIREMENTS  TRACEABILITY  MATRIX 

NAS- 1000 

Data  Management  Mode  - 

•  Product  reception  log,  alert  log,  product  dissemination  log  and 
system  operations  log 

•  Retention  and  database  update  functions 

1-01 

A,  T,  D 

2.4 

Backup/Degraded  Mode  Requirements 

1-01, 

E-01 

A,  T 

2.5 

Supportability  - 

•  NAS  System  Requirements  Specification  (SRS) 

•  FAA-OR-  6000.30B 

S-01  - 
S-02 

A 

2.5.1 

Availability  - 

•  Maintain  Availability  of  0.99982 

S-01- 

S-02 

A,  T 

2.5.2 

Reliability  - 

•  MTBF  of  2,777  hours 

S-01 

A,  T 

2.5.3 

Maintainability  - 

•  MTTR  shall  be  no  more  than  0.5  hours 

S-01, 

1-01 

A 

2.5.4 

Frequency  of  Maintenance  Actions  - 
•  limited  to  three  per  year 

S-01, 

1-01 

A 

NAS  ???? 

Backup  Power  Source 

1-01 

A 

5.0 

Support  Concept 

•  Status  of  External  Interfaces 

•  Resource  Utilization 

•  Errors 

•  Reporting 

•  NAS  Infrastructure  Management  System  (NIMS) 

•  Housekeeping  Functions 

1-01, 

E-01, 

D-03, 

S-01- 

S-02 

A 

A-2 


3.3 

Planned  Life  Cycle  -  Evolutionary  communication  system  that  will 
need  to  be  upgraded  as  FAA  weather  requirements  change 

E-04 

N/A 

NAS  ???? 

Provide  Upgradable  Throughput  rate 

1-01, 

E-04 

A 

NAS  IRS  VERIFICATION  REQUIREMENTS  TRACEABILITY  MATRIX 

NAS-IR- 

90029414 

NWSTG  /  FBWTG  IRD 

1-01 

A,  T,  D 

NAS-IR- 

94142514 

FBWTG  /  ITWS  IRD 

1-01 

A,  T,D 

NAS-IR- 

XXXXXX 

FBWTG  /  WARP  IRD 

1-01 

A,  T,  D 

NAS  OPERATIONAL  VERIFICATION 

MOE 

Measure  of  Effectiveness 

E-01  - 
E-04 

A 

MOS 

Measure  of  Suitability 

A 

MOP 

Measure  of  Performance 

1-01 

A 

A-3 


APPENDIX  B 


OT&E  TEST  EVALUATION  MATRIX 


OT&E  TEST  EVALUATION  MATRIX 


USER  OPERATIONAL  TEST: 

DATE: 

o 

B 

S 

E 

R 

j  V 

A 

N 

C 

E 

EVALUATION 

CRITERIA 

D 

I 

S 

c 

R 

E 

P 

A  ^ 

N  " 

C 

Y 

*  REMARKS 

DOCUMENTATION 

Completeness 

Availability 

TRAINING 

Effectiveness 

Hands-On 

HUMAN  FACTORS 

Error  Detection 

Safety 

LOGISTICS 

Expediency 

Turn-Around 

SYSTEM  MAINTENANCE 

Expediency 

Turn-Around 

SECURITY 

Expediency 

Turn-Around 

*ALL  failures/discrepancies  require 

explanation  in  remarks  column  SIGNATURE: 


Impact  ( Circle  One )  HIGH  MEDIUM  LOW  NONE 


B-l 


APPENDIX  C 


PROBLEM  TROUBLE  REPORT 


Solution:  Priority  (select  one): 

1  =  high  2  =  medium  3  =  low 


Severity  Code  (select  one): 

1  =  Critical  3  =  Non-Critical 

2  =  Critical  /  4  =  Cosmetic  / 

Work  Around  Insignificant  (explain) 

i _ _ _ 

STATUS  HISTORY 


Date  I  Status  Updated  By 


